feat: add LLM-generated release notes for GitHub releases#423
Merged
Conversation
- New script `scripts/gen-release-summary.sh`: Generates user-friendly editorialized release notes using Claude Code (sandboxed with read-only tools) - Updated `tasks/release-plz`: Integrates LLM generation into the release process, replacing raw git-cliff output with editorialized content - Updated `.github/workflows/publish-cli.yml`: Extracts release notes from CHANGELOG.md instead of regenerating Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add prompt instruction and runtime validation to prevent LLM from generating "## [" patterns that would corrupt changelog processing: - Added explicit instruction in prompt to never use "## [" - Added grep validation that rejects output containing the pattern - Exits with error if pattern detected Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use printf-based prompt construction to avoid backtick command substitution - Add instructions for PR links and documentation links in release notes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…releases - gen-release-summary.sh: concise 1-paragraph + bullets for CHANGELOG.md - gen-release-notes.sh: rich detailed notes for GitHub releases - Update publish-cli.yml to generate rich notes at release time Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
LLM-generated changelog content should only be created after the release PR is merged, not during PR creation. Now: - release-plz uses git-cliff directly for CHANGELOG.md - LLM editorialization only happens in release.yml for GitHub releases - Removes gen-release-summary script (no longer needed) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add fetch-depth: 0 to checkout step so git describe and git cliff have access to full git history - Validate Claude output is non-empty before using it, exit with error if empty so workflow falls back to CHANGELOG.md Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The PR title already contains the version, so strip the redundant version header (## [version] - date) from the PR body. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Log version, previous version, and changelog length before calling Claude - Capture stderr from Claude CLI and display on failure - Helps diagnose why LLM generation fails in CI Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Avoid polluting release notes output by redirecting stderr to a temp file instead of merging with stdout (2>&1). This prevents Claude CLI diagnostic messages from appearing in the release notes. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds LLM-generated release notes for GitHub releases while maintaining the existing raw git-cliff output in CHANGELOG.md. The key change is separating changelog generation for repository history from rich, user-friendly release notes for GitHub releases.
Changes:
- New script
gen-release-notes.shgenerates enhanced release notes using Claude Code API - Updated release workflow to generate LLM notes with fallback to CHANGELOG.md
- Modified release-plz task to use raw git-cliff for PR body and strip version headers
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| scripts/gen-release-notes.sh | New script that calls Claude API to generate user-friendly release notes from git-cliff changelog |
| tasks/release-plz | Updated to use raw git-cliff output and strip version headers from PR body |
| .github/workflows/publish-cli.yml | Added Claude CLI installation, LLM release notes generation with fallback logic |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| --notes "$(git cliff --latest --strip all)" | ||
| --notes-file /tmp/release-notes.txt | ||
| env: | ||
| ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} |
There was a problem hiding this comment.
The ANTHROPIC_API_KEY secret is only used in the conditional check at line 39 but not passed to the gen-release-notes.sh script execution. The script won't have access to the API key needed for the Claude CLI call. Add the ANTHROPIC_API_KEY to the environment when executing the gen-release-notes.sh script.
| gh pr create --title "chore: release $version" --body "$changelog" --label "release" || | ||
| gh pr edit --title "chore: release $version" --body "$changelog" | ||
|
|
||
| if [[ "$(gh pr list --label release)" == "" ]]; then |
There was a problem hiding this comment.
String comparison with command substitution can fail if the command output contains special characters or whitespace. Use -z test with command substitution or check the exit code. Replace with: if ! gh pr list --label release --json number -q '.[0].number' &>/dev/null; then
Suggested change
| if [[ "$(gh pr list --label release)" == "" ]]; then | |
| if ! gh pr list --label release --json number -q '.[0].number' &>/dev/null; then |
Comment on lines
+38
to
+46
| # Generate rich release notes using LLM | ||
| if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then | ||
| ./scripts/gen-release-notes.sh "$TAG_NAME" "$PREV_TAG" >/tmp/release-notes.txt || { | ||
| echo "LLM generation failed, falling back to CHANGELOG.md" | ||
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | ||
| } | ||
| else | ||
| echo "ANTHROPIC_API_KEY not set, using CHANGELOG.md" | ||
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt |
There was a problem hiding this comment.
The awk command to extract changelog entries is duplicated. Extract this into a variable or function to improve maintainability and ensure consistency between the fallback and no-API-key paths.
Suggested change
| # Generate rich release notes using LLM | |
| if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then | |
| ./scripts/gen-release-notes.sh "$TAG_NAME" "$PREV_TAG" >/tmp/release-notes.txt || { | |
| echo "LLM generation failed, falling back to CHANGELOG.md" | |
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | |
| } | |
| else | |
| echo "ANTHROPIC_API_KEY not set, using CHANGELOG.md" | |
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | |
| extract_changelog_notes() { | |
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | |
| } | |
| # Generate rich release notes using LLM | |
| if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then | |
| ./scripts/gen-release-notes.sh "$TAG_NAME" "$PREV_TAG" >/tmp/release-notes.txt || { | |
| echo "LLM generation failed, falling back to CHANGELOG.md" | |
| extract_changelog_notes | |
| } | |
| else | |
| echo "ANTHROPIC_API_KEY not set, using CHANGELOG.md" | |
| extract_changelog_notes |
Comment on lines
+36
to
+46
| TAG_NAME="${{ github.ref_name }}" | ||
| PREV_TAG="$(git describe --tags --abbrev=0 "$TAG_NAME^" 2>/dev/null || echo "")" | ||
| # Generate rich release notes using LLM | ||
| if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then | ||
| ./scripts/gen-release-notes.sh "$TAG_NAME" "$PREV_TAG" >/tmp/release-notes.txt || { | ||
| echo "LLM generation failed, falling back to CHANGELOG.md" | ||
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | ||
| } | ||
| else | ||
| echo "ANTHROPIC_API_KEY not set, using CHANGELOG.md" | ||
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt |
There was a problem hiding this comment.
The awk command to extract changelog entries is duplicated. Extract this into a variable or function to improve maintainability and ensure consistency between the fallback and no-API-key paths.
Suggested change
| TAG_NAME="${{ github.ref_name }}" | |
| PREV_TAG="$(git describe --tags --abbrev=0 "$TAG_NAME^" 2>/dev/null || echo "")" | |
| # Generate rich release notes using LLM | |
| if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then | |
| ./scripts/gen-release-notes.sh "$TAG_NAME" "$PREV_TAG" >/tmp/release-notes.txt || { | |
| echo "LLM generation failed, falling back to CHANGELOG.md" | |
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | |
| } | |
| else | |
| echo "ANTHROPIC_API_KEY not set, using CHANGELOG.md" | |
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | |
| extract_changelog() { | |
| awk '/^## \[/{if(found) exit; found=1} found{print}' CHANGELOG.md >/tmp/release-notes.txt | |
| } | |
| TAG_NAME="${{ github.ref_name }}" | |
| PREV_TAG="$(git describe --tags --abbrev=0 "$TAG_NAME^" 2>/dev/null || echo "")" | |
| # Generate rich release notes using LLM | |
| if [[ -n "${ANTHROPIC_API_KEY:-}" ]]; then | |
| ./scripts/gen-release-notes.sh "$TAG_NAME" "$PREV_TAG" >/tmp/release-notes.txt || { | |
| echo "LLM generation failed, falling back to CHANGELOG.md" | |
| extract_changelog | |
| } | |
| else | |
| echo "ANTHROPIC_API_KEY not set, using CHANGELOG.md" | |
| extract_changelog |
# Conflicts: # scripts/gen-release-notes.sh
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #423 +/- ##
=======================================
Coverage 48.20% 48.20%
=======================================
Files 47 47
Lines 6163 6163
Branches 6163 6163
=======================================
Hits 2971 2971
Misses 1557 1557
Partials 1635 1635 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Merged
jdx
pushed a commit
that referenced
this pull request
Jan 19, 2026
### 🚀 Features - **(release)** add LLM-generated prose summary to release notes by [@jdx](https://github.com/jdx) in [#421](#421) - add LLM-generated release notes for GitHub releases by [@jdx](https://github.com/jdx) in [#423](#423) - add spec lint command by [@jdx](https://github.com/jdx) in [#430](#430) ### 🐛 Bug Fixes - replace unsafe path unwrap chains with proper error handling by [@jdx](https://github.com/jdx) in [#424](#424) - pass positional args through to executed scripts by [@jdx](https://github.com/jdx) in [#425](#425) - replace unimplemented!() with proper errors for unsupported shells by [@jdx](https://github.com/jdx) in [#432](#432) - update claude CLI model and add bypassPermissions by [@jdx](https://github.com/jdx) in [#435](#435) ### 🚜 Refactor - remove unused double-shebang support by [@jdx](https://github.com/jdx) in [#426](#426) - replace once_cell with std::sync::LazyLock by [@jdx](https://github.com/jdx) in [#428](#428) - improve code quality with safety and lint fixes by [@jdx](https://github.com/jdx) in [#427](#427) ### ⚡ Performance - use Arc for flag/arg keys in ParseOutput to reduce cloning by [@jdx](https://github.com/jdx) in [#422](#422) ### 🔍 Other Changes - update insta snapshots to newer format by [@jdx](https://github.com/jdx) in [#429](#429) - fix legacy inline snapshot format warnings by [@jdx](https://github.com/jdx) in [#433](#433) - replace TODO with doc comment for subcommand_lookup by [@jdx](https://github.com/jdx) in [#434](#434) ### 📦️ Dependency Updates - update actions/setup-node digest to 6044e13 by [@renovate[bot]](https://github.com/renovate[bot]) in [#419](#419) - replace dependency @tsconfig/node22 with @tsconfig/node24 by [@renovate[bot]](https://github.com/renovate[bot]) in [#418](#418)
Merged
tmeijn
pushed a commit
to tmeijn/dotfiles
that referenced
this pull request
Jan 19, 2026
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [usage](https://github.com/jdx/usage) | minor | `2.12.0` → `2.13.1` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>jdx/usage (usage)</summary> ### [`v2.13.1`](https://github.com/jdx/usage/blob/HEAD/CHANGELOG.md#2131---2026-01-19) [Compare Source](jdx/usage@v2.13.0...v2.13.1) ##### 🐛 Bug Fixes - use correct PowerShell casing in enum variant by [@​jdx](https://github.com/jdx) in [#​438](jdx/usage#438) ### [`v2.13.0`](https://github.com/jdx/usage/blob/HEAD/CHANGELOG.md#2130---2026-01-19) [Compare Source](jdx/usage@v2.12.0...v2.13.0) ##### 🚀 Features - **(release)** add LLM-generated prose summary to release notes by [@​jdx](https://github.com/jdx) in [#​421](jdx/usage#421) - add LLM-generated release notes for GitHub releases by [@​jdx](https://github.com/jdx) in [#​423](jdx/usage#423) - add spec lint command by [@​jdx](https://github.com/jdx) in [#​430](jdx/usage#430) - add PowerShell completion support by [@​jdx](https://github.com/jdx) in [#​431](jdx/usage#431) ##### 🐛 Bug Fixes - replace unsafe path unwrap chains with proper error handling by [@​jdx](https://github.com/jdx) in [#​424](jdx/usage#424) - pass positional args through to executed scripts by [@​jdx](https://github.com/jdx) in [#​425](jdx/usage#425) - replace unimplemented!() with proper errors for unsupported shells by [@​jdx](https://github.com/jdx) in [#​432](jdx/usage#432) - update claude CLI model and add bypassPermissions by [@​jdx](https://github.com/jdx) in [#​435](jdx/usage#435) ##### 🚜 Refactor - remove unused double-shebang support by [@​jdx](https://github.com/jdx) in [#​426](jdx/usage#426) - replace once\_cell with std::sync::LazyLock by [@​jdx](https://github.com/jdx) in [#​428](jdx/usage#428) - improve code quality with safety and lint fixes by [@​jdx](https://github.com/jdx) in [#​427](jdx/usage#427) ##### ⚡ Performance - use Arc for flag/arg keys in ParseOutput to reduce cloning by [@​jdx](https://github.com/jdx) in [#​422](jdx/usage#422) ##### 🔍 Other Changes - update insta snapshots to newer format by [@​jdx](https://github.com/jdx) in [#​429](jdx/usage#429) - fix legacy inline snapshot format warnings by [@​jdx](https://github.com/jdx) in [#​433](jdx/usage#433) - replace TODO with doc comment for subcommand\_lookup by [@​jdx](https://github.com/jdx) in [#​434](jdx/usage#434) ##### 📦️ Dependency Updates - update actions/setup-node digest to [`6044e13`](jdx/usage@6044e13) by [@​renovate\[bot\]](https://github.com/renovate\[bot]) in [#​419](jdx/usage#419) - replace dependency [@​tsconfig/node22](https://github.com/tsconfig/node22) with [@​tsconfig/node24](https://github.com/tsconfig/node24) by [@​renovate\[bot\]](https://github.com/renovate\[bot]) in [#​418](jdx/usage#418) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi44NC4yIiwidXBkYXRlZEluVmVyIjoiNDIuODQuMiIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiUmVub3ZhdGUgQm90IiwiYXV0b21hdGlvbjpib3QtYXV0aG9yZWQiLCJkZXBlbmRlbmN5LXR5cGU6Om1pbm9yIl19-->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds LLM-generated release notes for GitHub releases while keeping CHANGELOG.md using raw git-cliff output.
Changes
scripts/gen-release-notes.sh: Generates rich user-friendly release notes using Claude Code for GitHub releasestasks/release-plz: Uses raw git-cliff for CHANGELOG.md (no LLM generation)publish-cli.yml:fetch-depth: 0for full git historyHow it works
release-plzgenerates a release PR with raw git-cliff changeloggen-release-notesto generate rich LLM notesError Handling
Security
The Claude Code agent is sandboxed with
--allowedTools "Read,Grep,Glob"(no Bash, Edit, Write).Test plan
🤖 Generated with Claude Code
Note
Enhances robustness of
scripts/gen-release-notes.shto keep release notes output clean and actionable.mktempand cleans it up with atrapWritten by Cursor Bugbot for commit 05207e1. This will update automatically on new commits. Configure here.